The personal computer has recently adopted multimedia applications which include audio and video information. Several multimedia groups have established and proposed standards for compressing/encoding and decompressing/decoding the audio and video information. MPEG I and II, established by the Motion Picture Expert Group, are the most widely accepted international standards in the field of the multimedia PC applications. Other standards are JPEG and Motion JPEG established by the Joint Photographic Expert Group. FIG. 1A is a diagram of an MPEG audio and video decoder 120 performs decompression of the video and/or audio data has been compressed and coded according to the MPEG algorithm. The system decoder 110 reads the encoded MPEG data stream 101 having interlaced compressed video and/or audio data, and generates necessary timing information; Video Presentation Time Stamp (VPTS) 104; System Clock Reference (SCR) 105 which is also referred to as system time clock (STC); Audio Presentation Time Stamp (APTS) 106; and separated video encoded bit streams 102 and audio encoded bit streams 103. The video decoder 111 decompresses the video data stream 102 and generates a decompressed video signal 107. The audio decoder 112 decompresses the audio data stream 103 and generates the decompressed audio signal 108. The decompressed video signal 107 is coupled to a PC monitor or other type of display unit, while the decompressed audio signal 108 is coupled to an audio speaker or other audio generation means.
The MPEG encoded/compressed data stream may contain a plurality of encoded/compressed video data packets or blocks and a plurality of encoded/compressed audio data packets or blocks. An MPEG encoder encodes/compresses the video packets based on video frames, also referred to as pictures. These pictures or frames are source or reconstructed image data consisting of three rectangular matrices of multiple-bit numbers representing the luminance and two chrominance signals. FIGS. 2A-2C illustrate the type of encoded/compressed video frames that are utilized. FIG. 2A depicts an Intra-frame or I-type frame 200. The I-type frame or picture is a frame of video data that is coded without using information from the past or the future and is utilized as the basis for decoding/decompression of other type frames. FIG. 2B is a representation of a Predictive-frame or P-type frame 210. The P-type frame or picture is a frame that is encoded/compressed using motion compensated prediction from an I-type or P-type frame of its past, in this case, I.sub.1 200. That is, a previous frame is used to encode/compress a present given frame of video data. 205a represents the motion compensated prediction information to create a P-type frame 210. FIG. 2C depicts a Bi-directional-frame or B-type of frame 220. The B-type frame or picture is a frame that is encoded/compressed using a motion compensated prediction derived from the I-type reference frame (200 in this example) or P-type reference frame in its past and the I-type reference frame or P-type reference frame (210 in this example) in its future or a combination of both. B-type frames are usually inserted between I-type frames or P-type frames. FIG. 2D represents a group of pictures in what is called display order I.sub.1 B.sub.2 B.sub.3 P.sub.4 B.sub.5 P.sub.6. FIG. 2D illustrates the B-type frames inserted between I-type and P-type frames and the direction which motion compensation information flows.
Motion compensation refers to using motion vectors from one frame to the next to improve the efficiency of predicting pixel values for encoding/compression and decoding/decompression. The method of prediction uses the motion vectors to provide offset values and error data that refer to a past or a future frame of video data having decoded pixel values that may be used with the error data to compress/encode or decompress/decode a given frame of video data.
The capability to decode/decompress P-type frames requires the availability of the previous I-type or P-type reference frame and the B-type frame requires the availability of the subsequent I-type or P-type reference frame. For example, consider the encoded/compressed data stream to have the following frame sequence or display order:
I.sub.1 B.sub.2 B.sub.3 P.sub.4 B.sub.5 P.sub.6 B.sub.7 P.sub.8 B.sub.9 B.sub.10 P.sub.11 . . . P.sub.n-3 B.sub.n-2 P.sub.n-1 I.sub.n.
The decoding order for the given display order is:
I.sub.1 P.sub.4 B.sub.2 B.sub.3 P.sub.6 B.sub.5 P.sub.8 B.sub.7 P.sub.11 B.sub.9 B.sub.10 . . . P.sub.n-1 B.sub.n-2 I.sub.n.
The decoding order differs from the display order because the B-type frames need future I or P-type frames to be decoded. P-type frames require that the previous I-type reference frame be available. For example, P.sub.4 requires I.sub.1 to be decoded such that the encoded/compressed I.sub.1 frame needs to be available. Similarly, the frame P.sub.6 requires that P.sub.4 be available in order to decode/decompress frame P.sub.6. B-type frames, such as frame B.sub.3, require a past and future I or P-type reference frames, such as P.sub.4 and I.sub.1 in order to be decoded. B-type frames are inserted frames between I-type, P-type, or a combination during encoding and are not necessary for faithful reproduction of an image. The frames before an I-type frame, such as P.sub.n-1 in the example, are not needed to decode an I-type frame, and no future frames require P.sub.n-1 in order to be decoded/decompressed.
Generally, every second of motion requires at least two I-type frames. I-type frames have a sampling rate of 150 kilobits per second, whereas P-type frames have a sampling rate of 50 kilobits per second.
The display process may be slower than the decoding process. For example, a 240.times.16 picture requires 3072 clock cycles to decode(76.8us at 40Mhz); it takes 200us to display 16 lines of video data at a 75 Hz refresh rate (13us.times.16=200us). The video frames are buffered before being displayed. There is usually a one frame delay between display and decoding. The difference between display and decoding leads to a condition known as tearing. Tearing occurs when the display frame is overwritten by the decoded frame.
FIG. 1B depicts tearing. A decoded/decompressed frame 132 of data representing the image of a closed door 133 is currently stored in a buffer 135. This decode/decompressed frame is currently being displayed on display unit 140. During this display period another decoded/decompressed frame 130 with data representing the image of an open door 131 is stored in buffer 135. The display unit 140 will now start displaying using information from the new frame now stored in 135. The result is a partial display of the first stored image 141 and partial display of the new stored image 142.
The various methods of combining the MPEG stream with the normal video stream can be seen from FIG. 7A-C.